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ABSTRACT 



This thesis approaches the question of ‘How much testing is enough?” by 
formulating a model for the combat situation in which the weapon (e.g., missile) will be 
used. Methods of Bayesian statistics are employed to allow the decision maker to benefit 
from prior information gained in the testing of similar systems by forecasting the 
operational gain from acceptance. A Microsoft Excel V7.0 spreadsheet serves as the user 
interface, and Visual Basic for Applications, Excel’s built in macro-language, is the 
language used to produce the source code. The methodology accommodates two different 
tactical usages for the missile: a single shot, or a salvo of two shots. The missile might be 
acceptable if used in the two-shot salvo mode, but not in the single shot mode, and this 
would imply a greater cost per mission. In the end the missile might not be judged cost- 
effective as compared to a competitive system. If the model proposed is (or can become) 
adequate much can be calculated/estimated before any operational tests are made. This 
could assist in economizing on operational testing. 
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THESIS DISCLAIMER 



The reader is cautioned that models and computer programs developed in this 
research may not have been exercised for all cases of interest. While every effort has been 
made, within the time available, to ensure that the models provide accurate results and the 
programs are free of computational and logic errors, they must be further validated and 
verified. The completion of these tasks is left for further research. Any application of 
these models and programs without additional validation and verification is at the risk of 
the user. 
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EXECUTIVE SUMMARY 



As a result of the end of the Cold War, the size and structure of the United States 
military continues to undergo serious scrutiny. Although this is a dynanruc process, the 
exact outcome of which is unknown, it is clear that in the future there will be significant 
reductions and changes in force structure, and a subsequent decrease in requirements for 
the generation of large numbers of new weapon systems. 

This trend will impact Testing and Evaluation programs, in that fewer dollars will 
be allocated for the test and evaluation infrastructure and process. As with other aspects 
of the acquisition process, the amount of testing performed should be determined by a 
cost-effectiveness measure. This can be described as a balance between the expense of 
testing and the reduction in risk for fielding the system gained by testing. One of the more 
expensive components of weapon system testing is that which involves destructive testing 
such as firing missiles or detonating ordinance. Such actions are required to provide some 
verification of the systems’ operational effectiveness and reliability. The need for testing 
in an operational environment adds further to the expense and complexity of the system 
evaluation. Clearly, testing methods which balance the need for economical efficiency and 
statistical integrity must be explored in order for the war fighter to continue to receive 
systems that are both effective and suitable, with firm attention to cost and value added. 

This thesis presents a formal analytical process for arriving at a number, t*, of 
missiles (or other expendable items) to test out of a finite lot or ‘buy” of m. It does so by 
estimating the operational combat utility of the missile, given acceptance. This means that 
testing focuses on adding operational value, rather than on simply reducing uncertainty, as 
simple hypothesis testing procedures tend to do. The methodology accommodates two 
different tactical usages for the missile: a single shot, or a salvo of two shots. The missile 
might be acceptable if used in the two-shot salvo mode, but not in the single shot mode, 
and this would imply a greater cost per mission. In the end the missile might not be 
judged cost-effective as compared to a competitive system. If the model proposed is (or 






can become) adequate much can be calculated/estimated before any operational tests are 
nade. This could assist in economizing on operational testing. 

The method proposed here is a suggestion and an approach; it is not a finished 
product. Various questions must be answered before the approach is practical. For 
example: how does one specify the prior (parameters) and the probability of successful 
opponent retaliation? Answer: from organizational experience with analogous systems, 
and from distilled expert judgment. Also, what to do with a system that is rejected (after 
testing an appropriate number of times and getting fewer than minimum number of 
successes)? The model does not attempt to address the choice of whether to end the 
program, or to look for particular faults that caused the deficiency and attempt to correct 
them. This choice is situation-specific, but if the system capability is needed and the faults 
are identifiable and rectifiable at reasonable cost then the latter course is attractive. 
Careful retrospective analysis of the test conditions is always important, whatever the 
outcome. 

In summary it is argued that some organized and defensible test planning and 
decision aiding process is needed by the Operational Testing community. The present 
approach is a proposed step on the path to filling that need. 
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I. INTRODUCTION 



A. BACKGROUND 

As a result of the end of the Cold War, the size and structure of the United States 
military continues to undergo serious scrutiny. Although this is a dynamic process, the 
exact outcome of which is unknown, it is clear that in the future there will be significant 
reductions and changes in force structure, and a subsequent decrease in requirements for 
the generation of large numbers of new weapon systems. 

This trend will impact Testing and Evaluation programs, in that fewer dollars will 
be allocated for the test and evaluation infrastructure and process. As with other aspects 
of the acquisition process, the amount of testing performed should be determined by a 
cost-effectiveness measure. This can be described as a balance between the expense of 
testing and the reduction in risk for fielding the system gained by testing. One of the more 
expensive components of weapon system testing is that which involves destructive testing 
such as firing missiles or detonating ordinance. Such actions are required to provide some 
verification of the systems’ operational effectiveness and reliability. The need for testing 
in an operational environment adds further to the expense and complexity of the system 
evaluation. Clearly, testing methods which balance the need for economical efficiency and 
statistical integrity must be explored in order for the war fighter to continue to receive 
systems that are both effective and suitable, with firm attention to cost and value added. 

B, CURRENT THINKING CONCERNING UNCERTAINTY 

When a test is designed it is set up to answer a decision maker’s question such as, 
‘How good is this?” or ‘Is it better than the one we have?” The challenge for the test 
designer is to translate these questions into something that is testable. Typically this 
translation results in statistical hypothesis testing. A key consideration in this type of 
testing is the fact that the statistician will never tell you that the hypothesis has been 
proven correct. He will only tell you whether or not it has been rejected. That is to say, 
statistical hypothesis testing can lead to only two outcomes; the hypothesis is rejected, or 



1 



the hypothesis cannot be rejected. Statistical hypothesis testing never directly confirms the 
hypothesis being tested. It is evident that the formulation or wording of the hypothesis is 
critical to the information the testing produces. Furthermore, conventionally applied 
hypothesis testing does not take into account gains from making correct decisions, or 
losses from making incorrect ones. 

Statistical hypothesis testing has another unattractive quality in these days of 
shrinking budgets in that it does not allow the tester to take advantage of prior 
knowledge. Only the data gathered in the context of the particular test formally qualifies 
for analysis. Information available from other resources such as developmental testing, 
testing of similar systems, field experience and the like are formally disregarded. The 
reason this could be unattractive in today’s fiscal environment is obvious. Information 
gained from another similar system’s testing process may remove the need for at least a 
portion of testing of the new system and thereby conserve the relevant funds. 

C. ALTERNATIVE THINKING 

Bayes’ theorem gives a probabilistically-based rule for updating the degree of 
belief in a hypothesis H (i.e., the probability of H) given additional evidence E and 
background information (context) I; 



some proposition under the assumption that some other propositions are true. This would 
seem to imply that Bayesian methods require the addition of more assumptions to obtain 
results, which is true. The most important assumption made is that of the value of the 
‘j)rior,”the P{H|I) term in the equation above. Here lies the rub. Prior probabilities have 
been not only ignored but abhorred by some classical statisticians. This has been due 
chiefly to the frequently subjective nature of the prior probabilities. In practice there may 
well be sufficient domain knowledge to specify a prior. In general, prior probabilities can 
be assigned to any unknown parameters involved in a formulation. After additional data is 




( 1 . 1 ) 



Note that all the probabilities are conditional. They specify the degree of our belief in 
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obtained these priors can be updated to a posterior. Clearly, Bayesian methods can take 
advantage of knowledge from other sources that is relevant to the testing, and thereby 
potentially save cost and time during the test and evaluation phase of system acquisition. 

D. THESIS OBJECTIVE 

This thesis employs a Bayesian framework in order to address a simplified version 
of a decision problem in destructive testing. It approaches the question of ‘How much 
testing is enough?” by formulating a model for the combat situation in which the weapon 
(e.g., missile) will be used. It then employs sequential Bayesian thinking to infer the 
testing level needed to achieve optimal expected gain. Questions of risk are also 
addressed. 
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n. PROBLEM FORMULATION AND SOLUTIONS 



A. INTRODUCTION 

A new system is proposed for acquisition, possibly to replace or upgrade an 
existing system. This new system will be accepted provided it meets minimum 
requirements for effectiveness and suitability which will be demonstrated through testing. 
This testing will be performed to replicate the mission and environment to which the 
system is expected to be exposed, and the results of a single test will be binary: success or 
failure. For the purposes of this discussion we assume the system is a missile and that the 
testing is destructive. That is to say, the test will destroy the missile but the record of its 
performance will be retained. 

We begin with m missiles which arrive as a lot and some number, t (0<t<m), of 
them which are to be tested to determine whether or not the lot will be accepted. An 
optimization problem presents itself If a small number of missiles (2 for example) are 
tested, less information is gained but a larger number of missiles is available to the fleet for 
use, given that the lot is accepted. Conversely, if a large number of missiles (m-2 for 
example) is tested, much more information is gained but only a few of the weapons are 
available for the missions for which they were designed (in this case, only 2). The 
intention of this paper is to provide a formulation that links the information gained from 
testing to the effect that gathering that information has on the missile’s future 
performance. In addition, the possibility of informing the decision as to how the missile 
could be “best” employed, or the choice of a shooting policy, will be discussed. 

B. UNCERTAINTY AND OPERATIONAL GAIN 

Assume that each of the missiles has the same, constant, and unknown probability 
of success, p. The decision as to whether or not the missiles should be deployed to the 
fleet will be based on the value of p. An option is to take t of the m missiles and test them. 
The number of tests could range from 0 to m; the latter would of course result in perfect 
knowledge for the lot of missiles but would leave no weapons for the fleet to employ. 



5 



Models can be built to represent the missile as it would be used operationally. For 
example; when the weapon is launched at an opponent he may, if he is missed, return fire 
with a weapon of his own. Let Vw be the value of a win, meaning a weapon hit on the 
enemy and V) be the value of a loss, a weapon hit by the enemy; q is the probability that the 
enemy will have a successful counterfire given missile miss. For simplicity, assume the 
value of q is known. The resulting gain function is, after t tests, 

G,(p, t; m) = (m - t)[v^p - v,(l - p)q] (2. 1) 

and represents the total expected gain from fielding a missile with success probability p 
determined from t tests which leaves (m-t) possible engagements. The subscript on Gi 
indicates that one weapon at a time was fired at the enemy. A second model, that 
describes the policy of firing a 2-weapon salvo, has gain function 



G 2 (p,t;m) = - (1 - p)') - v,(l - p)'q 

[v.(l-(l-p)’)-V|(l-p)’q 



ra-t 



( 2 . 2 ) 

(2.2a) 



Where [xj denotes the largest integer less than or equal to x. We use the approximation 



in the thesis. Note that a shoot-look-shoot policy is not evaluated here. The latter, while 
possibly more economical of fielded missile inventory, may well increase the chance of 
successful counterfire by the opponent (a larger q value). 



C. THE DECISION RULE 

1. Probability of Success Known 

If the decision maker knows the value of p he might then opt not to test, meaning 
that t=0. He would then evaluate G(p, 0; m); if the result is sufficiently positive, 
(exceeds a gain threshold), he accepts the system and fields it, while if the result is not 
sufficiently positive he rejects the system, achieving a gain of zero. The acceptance gain 
threshold may be defined by a predetermined minimum value of gain or by a minimum 
improvement in gain relative to a preceding system. This use of gain functions is of course 
quite simplified but serves the purpose of illustrating the method and provoking further 
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thought. In any case the decision whether or not to accept a system reduces to whether or 
not the system’s probability of success, p, meets or exceeds some minimum threshold, p . 

For example, in the case of the gain function for Gi, it can be seen that in order for the 
system to have positive gain, we must have a p-value that exceeds p = v,q / (v^ + v,q). 

2. Uncertainty in p: Bayesian Approach to Acceptance, Given a Test 
The decision to test t leads to the acquisition of data which is assumed to be 
summarized entirely as s successes (s = 0,l,2,...,t). These data can now be used in the 
development of a likelihood for p by use of a binomial model, and, if p has a beta prior, a 
beta posterior for p: 

n(p;s,t) = B(a',p')p»'-'(l- pf ■' (2.3) 

where a' = a + s and P' = P + 1 - s . The values (a,P) characterize the prior density for 
p. This is the classical conjugate prior setup which is standard; see Appendix A for details 
of the derivation. If more investigation shows that an alternative prior is more appropriate 
it is straightforward to supply the necessary changes. Now a decision maker in possession 
of the posterior, equation (2.3), should use it to evaluate the expected gain. This is 
accomplished by computing the gain’s expected value with respect to the posterior 
probability distribution, equation (2.3). In the case of Gi, the expectation is linear and of 
the form, 

E[G,(p,t;m)|s,t] = (m - t)[v,,E[p|s,t] - v,(l - E[p|s,t])q], (2.4) 

representing the expected gain from a fielded system. An appropriate decision rule for this 
case is to field the system if Ej^Gi(p,t;m)|s,tj is positive; otherwise ‘fejecf’the system. 

This is mathematically equivalent to fielding the system if E[^G](p,t;m)|s,tj > g, where 
g might represent the gain from utilizing an alternative system. Because of the form of Gi 

this is also equivalent to fielding the system if s(t) > s(t) where s(t) represents the 
minimum number of successes out of t tests which results in a positive value for Gi (or a 
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value of g > 0). The derivation of s(t)for both the single-weapon and two- weapon salvo 
cases can be found in Appendix B. 



D. HOW MUCH TO TEST 

1. Single-Weapon Case 

The previous section indicates what decision to make given the test extent, t, and 
the number of resultant successes, s(t). Now take the position of the decision maker 
before any destructive testing is performed. He must consider testing to any level. That is 
to say, he must consider testing 1, 2, 3, up to m missiles and determining which of the 
values for t provides the best potential for gain from the remaining missiles. Of course the 
results of the testing must satisfy the decision rules previously stated for the system to be 
fielded. The prediction used depends on the binomial model and upon the prior, which is 
assumed to be the beta prior with parameters a and (3: 

n(p)=B(o,|3)p“-‘(l-pf'‘, (2.5) 



Conditionally upon p, 

P{s(t) = s|p,t} 



.sj 



pv-pr\ 



0< s< t. 



( 2 . 6 ) 



In order to predict s(t) simply remove the condition on p, using the prior; this encapsulates 
the decision maker’s uncertain knowledge at the time he must decide on t. Details are 
found in Appendix A. The resultant predictive distribution, b(s;t), is the beta-binomial 



b(s;t) = P{s(t) = s|t} = p 



f'l 


^ r(a+p) ^ ^ 


UJ 


t^r(a)T(p)J[ 



t-s)^ 



(2.7) 



Now the expectation of gain, E[Gi], depends on the value of s(t), which is unknown 
during the planning phase of the test but whose probability distribution is given by 
equation (2.7), a beta-binomial. Consequently, the value of future gain can be predicted 
by unconditioning on s: 
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( 2 . 8 ) 



E{E[G,(p,t;m)|s,t]} = (m-t) (v„+v,q)J] xb(x;t)-v,q XKx;t) 

x>s(l) X^s(l) 

This expected gain can be evaluated over the entire range of t in principle. 



2. Two-Weapon Salvo Case 

Maintaining the aforementioned position of the decision maker and the questions 
facing him, we address the case in which 2 weapons are fired sequentially with no delay 
between shots. The beta prior, equation (2.5) and predictive distribution for s(t), equation 
(2.7), remain unaltered from the single weapon case. Thus, the expectation of G 2 also 
depends on the value of s(t) and can be predicted by calculating 



■|e G^(p,t;m)|s(t),t]| = [ 



m-t 
2 . 



( 



-'w Sb(x;t)-(v^ + v,q) Z / „ o ^ 

x>s(t) ^x>s(t)y(cx+P+t+l)[a+p+tjJ 



(p+t— x+l)^P+t— x^ 



b(x;t) 



(2.9) 

The derivation of s(t)can be found in Appendix B. This expected gain can also be 
evaluated over the entire range of t and consideration should be given to starting with 
small values of t and proceeding as in the single weapon case. 



E. RISK OF ACCEPTANCE 



The value of t, denoted by t*, which results in the maximum value of the predicted 
gain is determined by numerical search as described previously. Given that during the 
testing at least s(t) successes were observed and that the system was subsequently 
accepted, we are now interested in the risk associated with this acceptance. Two possible 
measures of this risk are (a) the predicted probability of kill per engagement (e g., a 
successful engagement), and (b) the probability that the number of mission engagements 
result in fewer successes than some threshold, D. Such information supplements that 
given by the expected value of gain, which involves not just p but also the chosen costs or 
penalties v* , vi, and q. 
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1. Single-Weapon Case 

The predicted probability of kill per engagement given acceptance of the system 
emerges from the now familiar beta-binomial distribution, conditional on acceptance of the 
system, s(t) > s(t) and with the beta based on the prior parameters, a and (3: 



Let M(t) be the number of successful engagements from utilizing the m-t missiles 
in combat. The probability that M(t) will fall short of some minimum standard, D, for kills 
can be found in terms of a binomial. First, 



P{M(t) = k|p,s) = jp’'(l-p) 



m-t-k 



k = 0,1,2, ...,m - 1. 



( 2 . 11 ) 



Next, remove the condition on p using the beta posterior: 
p{M(t) = k|s) = 



'^m - 1^ 


r(^oc+p+t^ 


r(a+s+kjr(p-s+m-k] 


V k y 


T(a+s)r(P+t-s) 


r(a+p+m) 



( 2 . 12 ) 



and the condition on s, given acceptance, yields after simplification. 



p|M(t) = k| Acceptance! “ 



m-ft 
k 



rfa+p+t) I r(a+j+k)r(p-j+m-k) 



r(a+p+m] 



S 



r(a+j)r(p+t-j] 



(2.13) 



Finally, 

D 

P[M(t) < DjAcceptancej = ~ k| Acceptance}. (2.14) 

k=0 

Now the above, (2.14), can be evaluated at t = t* for various D-values. It may be judged 
that the probability of mission success is simply intolerably low despite the fact that 
expected gain is positive. In that case the acceptance criterion, s(t), may be adjusted, and 
attempts made to increase the intrinsic capability of the new missile system, which could 
encompass performance as well as tactical considerations. 
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2. Two-Weapon Salvo Case 

In this case the probability of kill per engagement is found by first deriving the 
expectation of the probability that both weapons fired in the salvo miss their intended 
target, (1-p)^. This expected probability of failure is found in a similar fashion to that of 
the expected probability of success using the beta posterior, described in the single- 



weapon case. The expected posterior probability of success is then E 1 - (l - p)' 



Details 



are available in Appendix A. The predicted probability of kill per engagement, like the 
single-weapon case, is conditional on acceptance of the system, s(t) > s(t) and with the 



prior distribution with parameters, a and |3: 

(p+t-s+l)(p+t-s)^ 




b(s;t) 



S^(t) 



(2.15) 



Let M(t) be the number of successful engagements from utilizing the m-t missiles 
in combat. Since the missiles will be expended in pairs, let N(m,t) represent the maximum 
number of possible engagements with the missiles that remain after testing which is given 

by the largest integer less than or equal to ^ denoted by 



m-t 



The probability that 



M(t) will fall short of some minimum standard, D, for kills can also be found in terms of a 
binomial: 



p{M(t) = klp,s(t)} = (1 - p)')"((l - k = 0,1,2,. ..,N(m,t). (2. 16) 



Removing the condition on p using the beta posterior (2.7) and the binomial theorem (See 
Appendix C), 



P{M(t) = k|s} = 



(2.17) 



N(m,t)' 

k 



lA k 



fk'] 



„ r(a+p+t) r(2N(m,t)-2k+2n+p+t-s) 



r(p+t-s) r(2N(m,t)-2k+2n+a+p+t)’ 



and the condition on s yields after simplification. 
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p|M(t) = k|Acceptance| = 



(2.18) 



II 



N(m,t)YkYj..r(“+F(2N(ni,t)-2k+2n+p+t-j) 



(t\ 




Vjy 


^ k j 



vny 



2k4-2n4“Cx+p+tJ 



I, 



t^r(g+j)r(p+t-j) 

r(a+p+t) 



k = 0,1,2,... N(m,t). 



Finally, 



u 

p|M(t) < D|Acceptance| = ^P(M(t) = k|Acceptance|. (2.19) 



k=0 



As with the single-weapon case, the above, (2.19), can be evaluated at t = t* for 
various D-values. It may be judged that the probability of mission success is simply 
unacceptably low despite the fact that expected gain is positive. In that case the 
acceptance criterion, s(t) may be adjusted, and attempts made to increase the intrinsic 
capability of the new missile system, which could encompass performance as well as 
tactical considerations. 
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m. NUMERICAL EXAMPLE 



A. INTRODUCTION 

Consider a lot of 30 missiles offered by a manufacturer who has performed 
developmental testing and is satisfied that the uncertainty in each missile’s kill probability 
per engagement can be described by a beta distribution with parameters a and P, equal to 
5 and 1 respectively. We assume (2.1) adequately models the system in its combat 
environment and that an expected gain of 0 or greater is sufficient for acceptance although 
a gain of 0 might indicate no improvement over a current system. We further assume the 
values for Vw, vi, and q are known to be 1, 5, and 0.75. The 1 and 5 serve to indicate that 
the (negative) value of losing one of our aircraft is 5 times that of causing the destruction 
of one of the opponent’s aircraft. The 0.75 indicates that if the enemy has opportunity to 
counterfire he will with 0.75 probability succeed in shooting down our aircraft. 

B. THE ACCEPTANCE POLICY 

Accepting the system based upon a minimum expected gain is mathematically 
equivalent to accepting the system if s(t) > s(t) . The value of s(t) is found by evaluating 
(B.6); 

v,q(a + PM)_^ 
v„+v,q 

_ 5(0.75)(5 + l + t) ^ 

1 + 5(0.75) 

Negative values of s(t)are set equal to 0 while all other values are rounded up to the 
nearest integer. This is consistent with both the idea that only non-negative numbers of 
successes can be observed and that s(t)is to provide a lower bound on acceptable gain 

which is itself restricted to values greater than or equal to 0. 
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Next s(t)and (2.7) are used to evaluate the predicted expected gain given by (2.8) 
for each value oft (O < t < 30) . 



e{e[g,(p, t;m)|s(t),t| Acceptance! = (m -t) 



(vw + Viq) ^ xb(x;t) - v,q 2^b(x;t) 

x^s(t) x>s(t) 



(3.2) 



= (30-t) 






r(5+x)r(i+t+x) 



r(6+t) 



Relevant values of t, s(t) , and predicted gain can be seen in Table 3.1. If the predicted 

gain were determined to be zero through the range of t, the system would be rejected 
without testing since there is at best no gain from the testing expenditure. Also, if the 
maximum expected gain is achieved with no testing, t = 0, then the evaluation of the risk 



t 


0 


1 


2 . 


3 


4 


5 


6 


7 


s(t) 


0 


1 


" 2 


3 


3 


4 


5 


6 


E[G,] 


6.25 


7.77 


8.13 


7.97 


7.94 


8.05 


7.95 


7.72 



Table 3.1. Example Results. 



associated with deployment of the system becomes more significant. Neither of these 
situations occur in this example. In this example maximum expected gain occurs when 2 
of the 30 missiles are tested and the remaining 28 missiles are deployed. The policy is to 
accept the missile if both of the tests are successful. It is noticeable that the expected gain 
has a somewhat flat or slowly-changing dependence on t, so a judgment that a few more 
tests are desirable, based on unmodeled considerations, does not incur a great change in 
expected gain. Extra tests might be conducted to reveal surprising system faults. 
Additional testing under different tactical conditions may very well be necessary. 

C. RISK ASSESSMENT 

We are now interested in the risk associated with acceptance of the missile system 
subsequent to observing at least s(t) successes during testing. The first measure of risk 
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considered is the predicted probability of kill per engagement given acceptance which can 
be obtained from (2. 10) in concert with (2.7). The same numerical example is used for 
illustration. 



V f g+x "l h( V t) ^ 5 + X f 2 ) r(6) r(5 + x)r(3 + x) 

p(t) fW ) — 



_ x=2 



x>s(t) 



8 vxy r( 5 ) r(8) 

^f2)r(6) r(5+x)r(3+x) 
r(5) r(8) 



= 0.875 



(3.3) 



The second measure of risk considers the probability that the number of mission 
engagements result in fewer successes than some threshold, D given acceptance. This step 
is accomplished by evaluating (2. 13) 

fm-t) ^ ^ ft) 

r(a+p+t) X . r(a+j+k)r(p-j+m-k) 

P{M(t) = k|Acceptance} = r(a+p+m) ‘ 



I 

j>s(t) 



r(a+j)r(p+t-j) 



(3.4) 



^28^ 

xky 



r(8)Sl r(5+x+Or(3i->:-k) 



i;Qr(5+x)r(3-x) 

X =2 vxy 



then simply sum the results for the cumulative distribution up to D. Table 3.2 contains an 



D 
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4 


12 


13 


14 


15 


24 


25 


26 


27 


28 


P { M = D } 


0 


0 


0 


0 


0 


.002 


.004 


.006 


.008 


.088 


.109 


.135 


.165 


.2 


P { M < D } 


0 


0 


0 


0 


0 


.007 


.012 


.017 


.025 


.391 


.501 


.635 


.8 


1 



Table 3.2. Risk Summary. 



illustrative portion of the results for this example. In this example there is a 50% 
probability of 25 or fewer successes out of 28, or a 50% chance of 25 or more successes 
out of 28. Such information enhances the decision maker’s intuition. 
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D. SUMMARY 



The example above demonstrates a formulation that links the information gained 
from testing to the effect that testing has on a particular measure of the performance of the 
system once fielded. Despite the fact that the example is simple, valuable information is 
gleaned from it. In this example some guidance is provided to an individual who might be 
deciding just how many of the 30 available missiles should be tested. While testing 2 of 
the 30 missiles may not be the answer, if one places some belief in the assumptions made 
at the beginning of the exercise it is apparent that the number to test is nearer 2 than a 
considerably larger number, such as 15. Furthermore, the expense has been minimal 
compared to that of expending weaponry under one set of tactical conditions. Additional 
missile testing under different conditions is needed to explore the range of possible combat 
environments, so it may be reassuring that a comparatively few tests are recommended for 
each of these explorations. 
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IV. SOFTWARE IMPLEMENTATION 



A. INTRODUCTION 

The methods developed in previous chapters are, even for the simplest cases, 
computationally tedious. This invites one to use a computer not only for the convenience 
but also for the speed. Convenience is lost to some degree when one must learn a 
programming language in order execute the type of calculations involved in this 
formulation. On the other hand, more and more people are becoming familiar with 
spreadsheets and their use, especially in the analytical community. Increasingly, 
spreadsheets available commercially are suitable for situations that once required a custom 
program written in a traditional programming language. One such spreadsheet is 
Microsoft’s Excel V7.0. Excel and its accompanying macro language , Visual Basic for 
Applications, were used in the production of this project’s software. The code for this 
application is located in Appendix D. The ‘feadme.txt” help file and software installation 
directions are located in Appendix E. 

B. OVERVIEW 

1. Input 

The user is presented with a window which prompts him for the necessary 
information (See Figure 4.1). The inputs for both the single-weapon and two-weapon 
salvo cases are handled in an identical manner. Default values are indicated with the 
purpose of avoiding program crashes caused by blank input fields. Once the user is 
satisfied with his input selection, he merely clicks the ‘Begin” button or presses the return 
key on his keyboard to initiate the computations. 
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Figure 4.1. Input User Interface. 



2. Output 

After the user initiates the computations and the algorithm has executed, a 
spreadsheet page is produced. The worksheet contains a plot of the predicted gain values 
over the range of t (Figure 4.2) and several tables. The tables display information 
regarding the relationship between t, E[Gi], and s(t); the optimal number of missiles to 
test, t*; and the values associated with the risk incurred if the system is fielded. 
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Figure 4.2. Typical Output Worksheet. 
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c. 



DEMONSTRATION 



The following serve to demonstrate the scope of this project’s software through 
various combinations of weapon cases and analysis outcomes. Although the combinations 
are not exhaustive, they illustrate sufficiently to express the breadth of outcome 
possibilities. 

1. Single-Weapon Case with Testing Recommended 

Consider a lot of 15 missiles whose prior probability of kill has a beta distribution 
with parameters a and p, equal to 5 and 1 respectively (Figure 4.2). The values of v«, vi 
and q are 1, 5, and 1 respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. The algorithm finds that maximum 
gain is achieved when 3 missiles are tested and an acceptance policy of 3 successes from 3 
trials is employed. See Figure 4.2 for these results as well as the risk of acceptance 
predictions. 

2. Two-Weapon Salvo Case with No Testing Recommended 

Consider a lot of 15 missiles whose prior probability of kill has a beta distribution 
with parameters a and P, equal to 5 and 1 respectively (Figure 4.3). The values of v^, vj 
and q are 1, 5, and 1 respectively. The values used for the other relevant parameters are 
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Figure 4.3. Two-Weapon Salvo Case with No Testing Recommended. 
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displayed in the output worksheet for convenience. Note that the cost values are the same 
as those in Figure 4.2. However this case envolves consideration for the two-weapon 
salvo where the penalty for accepting a system with a low value of p is small. 
Subsequently, the algorithm finds that maximum gain is achieved when 0 missiles are 
tested and suggests system acceptance without testing (it is likely that this would be 
overridden in practice). In this case the risk of acceptance analysis is particularly germane. 
See Figure 4.3 for these results. 

3. Single-Weapon Case with System Rejection 

Consider a lot of 1 5 missiles whose prior probability of kill has a beta distribution 
with parameters a and |3, equal to 5 and 1 respectively (Figure 4.4). The values of v„, vj 
and q are 1, 25, and 1 respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. The algorithm finds that maximum 
gain that could be achieved is no better than that gain which exists without the missile 
system being considered and suggests system rejection without testing. In this case the 
cost of loss is Vi = 25 as compared to vj = 5 in Figure 4.3. Acceptance of this system 




would result in gain which is at best zero and probably negative. In other words, 
acceptance of the system will place the decision maker in no better a position. See Figure 
4.4 for these results. 
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4. Two-Weapon Salvo Case with Testing Recommended 
Consider a lot of 15 missiles whose probability of kill has a beta distribution with 
parameters a and P, equal to 5 and 1 respectively (Figure 4.5). The values of v„, vi and q 
are 1, 35, and 1 respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. Note that testing is recommended 
here (accepting the system if 5 successes are observed in 5 tests) while testing is not 
recommended Figure 4.3. In this case the cost of a loss here is V| = 35 as compared to V| = 
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Figure 4.5. Two-Weapon Salvo Case with Testing Recommended. 



5 in Figure 4.3. It is reasonable that more care be taken in the present more costly case: 
the penalty for accepting a low value of p is much greater here than in the case of Figure 
4.3. The algorithm finds that maximum gain is achieved when 5 missiles are tested and an 
acceptance policy of 5 successes from 5 trials is employed. See Figure 4.5 for these 
results as well as the risk of acceptance predictions. 
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V. SUMMARY DISCUSSION 



This thesis presents a formal analytical process for arriving at a number, t*, of 
missiles (or other expendable items) to test out of a finite lot or ‘buy” of m. It does so by 
estimating the operational combat utility, called gain, of the missile, given acceptance. 
This means that testing focuses on adding operational value, rather than on simply 
reducing uncertainty, as simple hypothesis testing procedures tend to do. The 
methodology accommodates two different tactical usages for the missile: a single shot, or 
a salvo of two shots. The missile might be acceptable if used in the two-shot salvo mode, 
but not in the single shot mode, and this would imply a greater cost per mission. In the 
end the missile might not be judged cost-effective as compared to a competitive system. If 
the model proposed is (or can become) adequate much can be calculated/estimated before 
any operational tests are made. This could assist in economizing on operational testing. 

The method proposed here is a suggestion and an approach; it is not a finished 
product. Various questions must be answered before the approach is practical. For 
example; how does one specify the prior (parameters) and the probability of successful 
opponent retaliation? Answer; from organizational experience with analogous systems, 
and from distilled expert judgment. Also, what to do with a system that is rejected (after 
testing t* times and getting fewer than s(t*) successes)? The model does not attempt to 
address the choice of whether to end the program, or to look for particular faults that 
caused the deficiency and attempt to correct them. This choice is situation-specific, but if 
the system capability is needed and the faults are identifiable and rectifiable at reasonable 
cost then the latter course is attractive. Careful retrospective analysis of the test 
conditions is always important, whatever the outcome. 

In summary it is argued that some organized and defensible test planning and 
decision aiding process is needed by the OT test community. The present approach is a 
proposed step on the path to filling that need. 
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APPENDIX A. BINOMIAL BAYES 



The probability of a number of successes given a fixed number of trials is 
calculated using the binomial distribution. The usual parameters are n, the number of 
trials; s, the number of successes in n trials; and p, the probability of success for any of the 
single trials. In the case of the beta-binomial distribution the parameter p is assumed to be 
random having a beta distribution. After n trials have been executed determining s, the 
Bayes update of the beta prior for p is possible. 

A. THE BETA POSTERIOR 

Assuming a Beta prior for p. 




and binomial sampling (n trials, s successes), the likelihood function for p is 




(A.2) 



The Bayes posterior is found by multiplying the prior for p by the likelihood and 
normalizing 




(A.3a) 




(A.3b) 



Also, since the density must integrate to 1, 



C 




(A.4) 



Then, the beta posterior mean for p is 




(A. 5) 
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Later the expectation of (1-p)^ will be needed. It can be derived in a fashion similar to 
that above 



e[(1 - p)’|n,s] = B(aMi') j(l - p)"p“'-'(l - pf"'dp 

0 

e[(1 - P)’|n,s] = j p“'"(l - p)'’""'<lp 

where a' = a + s and |3' = |3 + 1 - s. Finally, the mean beta posterior is 

fFh - n'»"ln = (P + ^~s + l)(P + n-s) 

(a + p + n + l)(a + |5 + n) 



(A. 6a) 



(A. 6b) 



(A.7) 



B. THE PREDICTIVE DISTRIBUTION FOR THE NUMBER OF 
SUCCESSES 

This derivation takes advantage of the beta prior for p in the development of a 
predictive distribution for the number of successes, s, in t trials. A binomial model that 
utilizes the beta prior, (A.l), and that is conditional on p is formed 

P{s(t) = s|p,t} = Qp*(l- p)“'. (A.8) 



Removing the condition on p yields 

Ht) = j(3p‘(i - pr-^^p“-'(> - pf “p 

Let a' = a + sand P' = P + 1 - s and combine terms to obtain 





r(a + p) j- 


UJ 


r(o)r(|i)| 



and simplify to get 



Pis(t\ = dtl = + P) + s)^(P + ^ ~ s) 

lsjr(a)r(p) r(a + p + t) 



(A.9) 



(A. 10) 



(A. 11) 
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APPENDIX B. DERIVATION OF MINIMUM SUCCESS THRESHOLD 



This appendix describes and then derives an expression for s(t), the minimum 
number of successes needed during testing in order to accept the system being tested. 

A. INTRODUCTION 

Consider a decision policy in which a system must provide a minimum level of gain 
from testing and subsequent fielding, given that the system is accepted. Let this minimum 
gain be g. Because of the form of the gain functions, (2.1) and (2.2), used in this paper 

acceptance of the system based on observing at least a minimum number of successes 
during testing, s(t), is mathematically equivalent to acceptance of the system based on 
receiving a minimum gain, g. 

B. DERIVATION OF s(t) 

1. Single-Weapon Case 

For ease of discussion, a gain of 0 has been deemed the minimum value of gain 
which allows system acceptance in this paper, 

E[G,(p,t;m|s(t),t)]>0. (B.l) 

From (2.4) it is evident that 

(m - t)Jv„E[p|s(t),t] - v,q(l - E[p|s(t),t])] > 0 (B.2) 

implies that 

E[p|<t),tl > —5^. (B.3) 

‘ ' V, + v,q 

Next substituting the beta posterior (A. 5), 

iL±fW>_Ii3_ (B4) 

a + P + t v^+v,q 

and solving for s(t) gives 
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v,q(a + p + t) 

s(t) > a. 

Vw +v,q 



(B.5) 



-a 



(B.6) 



Since s(t) > s(t) for non-negative gain it follows that 

Vw +v,q 

where j" z] is the smallest non-negative integer greater than or equal to z. 

2. Two-Weapon Salvo Case 

Again for ease of discussion, a gain of 0 has been deemed to be in this paper, the 
minimum value of gain which allows system acceptance. 

E[G2(p,t;m|s(t), t)]>0. 

In a fashion similar to the single-weapon case, 

(m-t) 



- (vw + v,q)E[(l - p)'|<t),tj] > 0 



and substituting the beta posterior, (A.7), yields for the bracketed term 

(P + 1 - s + l)(p + 1 - s) 



Vw -(vw +v,q) 



(a -I- P + 1 -I- l)(a + P + 1) 



> 0 . 



(Note we are replacing 
Let 



m-t 



in (2.2) with 



m-t 



V 2 



•) 



^ ^ Vw(« + P + t+l)(a+P + t) 
Vw+v,q 



and let 



r = t - s. 



Now rearrange (B.9) to obtain 



(B.7) 



(B.8) 



(B.9) 



(B.IO) 



(B.ll) 



r^ + 



(p + r + l)(p + r)-A = 0 
^2P + l)r + ^P^ -I- P — a| = 0. 



(B.12) 
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The quadratic formula is now utilized to find the roots of (B.12) The root of 
interest, r*, will be the one that is real, positive and largest in magnitude. The roots can be 
found by computing 

-2p-l±^(2p + l)'-4(p'+p-A) 
r = j ^ (B.13) 

Finally, 

s(t) = max^t - |_r* (B. 14) 

where [xj is the largest integer less than or equal to x. 
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APPENDIX C. BINOMIAL THEOREM APPLICATION 



This appendix states the binomial theorem and discusses its application with regard 
to the derivation of a formulation for risk assessment in the two-weapon salvo case. 

A. THE BINOMIAL THEOREM 



r=0 






(C.l) 



B. APPLICATION OF BINOMIAL THEOREM 

Let M(t) be the number of successful engagements possible after t missiles have 



been tested and let N(m,t) = 



m -t 



represent the largest integer less than 



m -t 



The 



probability of k successes out of N(m, t) engagements conditional on p, the probability of 
success for a single missile, and s, the number of successes during testing of t missiles is 
given by 

P{M(t) = k|p,s} = -(1 - P)’)‘((l - (C.2) 

Applying the binomial theorem yields 

P{M(t) = k|p,s) = >)■((> - P)7(')‘’" (C.3) 



and reduces to 



p{M(t) = k|p.s) = - p)’) 



o\2N(m.t)-2k+2n 



(C.4) 



Removing the condition on p 
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P{M(t) = k|s} = 




m,t) 






(-i)"(0-p)') 



2N(m,t)-2k+2n 



r(a + P + t) „aH-s-ui 
rla + s^p + .-s)” (''P> 



(CJ) 



f 



\ 



N(m,t)"|^rk^ 

k j“oUJ^ ^ 



r(a+p+t) 
r(a + s)r(p+t - s) 



r(a + s)r(2N(m,t)-2k + 2n + P + t- s) 
r^2N(rn,t) ~ 2k + 2n + ct + P + t~s^ 
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APPENDIX D. VISUAL BASIC FOR APPLICATIONS CODE 



This appendix contains the Visual Basic for Applications code used in the spreadsheet 
implementation software. 

A. Gl.XLS, SINGLE WEAPON GAIN WORKBOOK 



LT John R. Gorman 
Spring 1997 

Test Length Decision Analysis 



Sub getJnputO 

This section gets input for vw, vl, q, a, b, & m. The main subroutine 
'is then called and executed. 

Dim DBoxBegin As Boolean 

DialogSheetsflnput Box").EditBoxes("input_vw").Text = ”1" 
DialogSheetsf’Input Box").EditBoxes(”input_vl").Text = "5" 
DialogSheetsf’Input Box").EditBoxes("input_q").Text = "1” 
DialogSheetsf’Input Box").EditBoxes("input_a").Text = "5" 
DialogSheetsf'Input Box").EditBoxes("input_b").Text = "1" 
DialogSheetsf’Input Box").EditBoxes("input_m").Text = "15" 
DBoxBegin = DialogSheets("lnput Box").Show 
If Not DBoxBegin Then Exit Sub 
Sheets("Results").Range("vw"). Value = _ 

DialogSheetsflnput Box").EditBoxes("input_vw”).Text 
Sheets("Results").Range("vr).Value = _ 

DialogSheetsC'Input Box").EditBoxes("input_vl").Text 
Sheets("Results").Range("q").Value = _ 

DialogSheetsC'Input Box").EditBoxes("input_q").Text 
Sheets("Results").Range("alpha").Value = _ 

DialogSheetsC'Input Box").EditBoxes("input_a").Text 
SheetsC’Results").Range("beta").Value = _ 

DialogSheetsC’Input Box").EditBoxes(''input_b").Text 
Sheets(''Results").Range("m").Value = _ 

DialogSheetsC’Input Box"). EditBoxes("input_m") .Text 
Call plot_gain 
End Sub 

Sub plot_gainQ 

' Calculate expected mean gain for single weapon shot, display 
' tables and plots. 

Application. ScreenUpdating = False 

Dim vw As Variant 

Dim vl As Variant 

Dim q As Variant 

Dim a As Variant 
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Dim b As Variant 
Dim t As Variant 
Dim s_bar As Variant 
vw = Range("vw").Value 
vl = Range("vl").Value 
a = Rangefalpha"). Value 
b = Range("beta").Value 
m = Range("m").Value 
q = Range("q").Value 
SheetsC'Resu Its"). Select 
Rangeft"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 

Selection.Clear 

Range("s_bar"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection.Clear 
SheetsC'Resu Its"). Select 
Range("E_G1").Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection.Clear 
For t = 0 To m 
Range("t"). Select 
ActiveCell.Offset(t, 0).Value = t 
s_bar= Application. RoundUp(_ 

Application. Max(0, ((vl * q * (a + b + t) / (vw + vl * q)) - a)), _ 

0 ) 

EG1 =0 

For j = s_bar To t 

bjl = Exp(Application.GammaLn(a + b)) * Exp(Application.GammaLn(a + j)) * 
Exp(Application.GammaLn(b + 1 - j)) 

bJ2 = Exp(Application.GammaLn(a)) * Exp(Application.GammaLn(b)) *_ 
Exp(Application.GammaLn(a + b + t)) 
bJ3 = Application. Combin(t, j) 
bj = bJ1 / bJ2 * bJ3 

EG1_1 = ((vw + vl * q) * (a + j) / (a + b + t)) - (vl * q) 

EG1 = EG1 + (EG1_1 * bJ) 

Next) 

EG1 = Application. Max(EG1 * (m - 1), 0) 

Range("E_G1"). Select 
ActiveCell.Offset(t, 0).Value = EG1 
ActiveCell.Offset(t, 1). Value = s_bar 
Next t 

Dim check_range As Range 
Dim optimalj As Integer 
Dim t_counter As Integer 
Dim optimal_gain As Variant 
Sheets("Results"). Select 
Range("E_G1").Activate 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Set check_range = Selection 
Range("max_gain”). Select 
ActiveCell.Value = Application. Max(check_range) 
optimal_gain = Application.Max(check_range) 

Range("E_G1"). Activate 
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For t_counter = 1 To m + 1 
If ActiveCell. Value = optimal_gain Then 
optimal_t = ActiveCell. Offset(0, -1). Value 
GoTo loop_exit 
End If 

ActiveCell. Offset(1, 0).Activate 
Next t_counter 
loop_exit: 

Range("t_star"). Value = optimal_t 

This routine calculates the risks associated with accepting the 
'system after testing. 

j 

Dim k As Variant 

Dim PM1 As Variant 

Dim PM2 As Variant 

Dim PM3 As Variant 

Dim PM As Variant 

Dim t_star As Variant 

Dim m_t As Variant 

Dim i As Variant 

Dim p_bar As Variant 

Dim p_bar1 As Variant 

Dim p_bar2 As Variant 

t_star= Range(”t_star").Value 

m_t = m - t_star 

s_bar_star = Application. Max(_ 

(Application. RoundUp((vl * q * (a + b + t_star) 
/ (vw + vl * q)) - a, 0)), 0) 
Range("s_bar_star").Value = s_bar_star 
p_bar = 0 
p_bar1 = 0 
p_bar2 = 0 

SheetsfResu Its”) .Select 
Range(”PM”). Select 

Range(ActiveCell, ActiveCell. End(xlDown)). Select 
Selection. Clear 
Range(”k”). Select 

Range(ActiveCell, ActiveCell. End(xlDown)). Select 
Selection. Clear 
Range(”PMD"). Select 

Range(ActiveCell, ActiveCell. End(xlDown)). Select 
Selection. Clear 
Range(”p_bar”). Select 
Selection. Clear 

If ((t_star= 0) And Not (optimal_gain = 0)) Then 
GoTo NoTest 

Elself (optimal_gain = 0) Then 
GoTo Reject 
Else 

For k = 0 To m_t 
PM2 = 0 
PM3 = 0 

PM1 = (Application. Combin(m_t, k) *_ 
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Exp(Application.GammaLn(a + b + t_star))) /_ 
Exp(Application.GammaLn(a + b + m)) 

For j = s_bar_star To t_star 

PM2 = PM2 + ((Application. Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j + k)) *_ 
Exp(Application.GamnnaLn(b- j + m - k)))) 

PM3 = PM3 + ((Application. Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammaLn(b + t_star - j)))) 

Next j 

PM = PM1 * (PM2 / PM3) 

Sheets("Results"). Select 
Range("PM").Select 
ActiveCell.Offset(k, 0).Activate 
ActiveCell.Value = PM 
Next k 

For j = s_bar_starTo t_star 

p_bar1 = p_bar1 + (a + j) * ((Application.Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammaLn(b + t_star- j)))) 
p_bar2 = p_bar2 + ((Application. Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammaLn(b + t_star - j)))) 

Next j 

p_bar = p_bar1 / ((a + b + t_star) * p_bar2) 

Range("p_bar").Select 
ActiveCell.Value = p_bar 
Range("p_bar_comment”). Select 
ActiveCell. Clear 
Range("PMD").Select 
ActiveCell. FormulaR1C1 = ’’=RC[-1]" 

For i = 1 To m_t 
ActiveCell. Offset(1 , 0).Activate 
ActiveCell. FormulaR1C1 = "=RC[-1]+R[-1]C" 

Next i 

Range("k").Select 
For i = 0 To m_t 
ActiveCell.Value = i 
ActiveCell. Offset(1, 0).Activate 
Next i 
End If 
GoTo Fini 
Reject: 

MsgBox "Reject System" 

Range("s_bar"). Select 

Range(ActiveCell, ActiveCell. End(xlDown)). Select 
Selection. Clear 

Range("s_bar_star").Value = "NA" 

Sheets("Results"). Select 
Range("p_bar").Select 
Selection. Clear 

Ra ng e (" p_ba r_co m ment") . Se I ect 
Selection. Clear 
Range("PM"). Select 
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ActiveCell.Value = "REJECT" 

ActiveCell.Offset(0, 1).Activate 
ActiveCell.Value = "SYSTEM" 

GoTo Fini 
NoTest: 

MsgBox "Optimal Gain Achieved with No Testing" 

SheetsC'Results"). Select 
Range("p_bar"). Select 
Selection. Clear 
ActiveCell.Value = a / (a + b) 

ActiveCell.Offset(0, 1).Value = Prior probability used since 0 tested" 
Range("k"). Select 
ActiveCell.Value = "NO" 

ActiveCell.Offset(0, 1). Activate 
ActiveCell.Value = "TESTING" 

ActiveCell.Offset(0, 1).Activate 
ActiveCell.Value = "DONE" 

Fini: 

SheetsC'Results"). Range("c2"). Select 
End Sub 

'This routine provides for the functions that occur on workbook opening. 

I 

Sub Auto_openO 
Call get_input 

ActiveWorkbook.Protect contents = True 
End Sub 



B. G2.XLS, TWO-WEAPON SALVO GAIN WORKBOOK 



LT John R. Gorman 
Spring 1997 

Test Length Decision Analysis 



Sub get_inputO 

'This section gets input for vw, vl, q, a, b, & m. The main subroutine 
'is then called and executed. 

Dim DBoxBegin As Boolean 

DialogSheets("lnput Box").EditBoxes("input_vw").Text = "1" 
DialogSheets("lnput Box").EditBoxes("input_vl").Text = "15" 
DialogSheets("lnput Box").EditBoxes("input_q").Text = "1" 
DialogSheetsflnput Box"). EditBoxes("input_a"). Text = "5" 
DialogSheetsflnput Box").EditBoxes("input_b").Text = "1" 
DialogSheets("lnput Box").EditBoxes("input_m").Text = "15" 
DBoxBegin = DialogSheets("lnput Box"). Show 
If Not DBoxBegin Then Exit Sub 
SheetsC'Results"). Range("vw").Value = _ 

DialogSheetsC'Input Box").EditBoxes("input_vw").Text 
Sheets("Results").Range("vl").Value = _ 
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DialogSheetsflnput Box").EditBoxes("input_vl").Text 
Sheets("Results").Range("q"). Value = _ 

DialogSheetsC'Input Box").EditBoxes("input_q").Text 
Sheets(”Results").Range("alpha").Value = _ 

DialogSheetsC’Input Box").EditBoxes("input_a").Text 
Sheets("Results").Range("beta"). Value = _ 

DialogSheetsC’Input Box"). EditBoxes("input_b"). Text 
Sheets(’’Results’’).RangeC’m’’).Value = _ 

DialogSheetsC’Input Box"). EditBoxes(’’input_m"). Text 
Call plot_gain 
End Sub 

Sub plot_gainO 

I 

’ Calculate expected mean gain for single weapon shot, display 
’ tables and plots. 

Application. Screenupdating = False 

Dim r1 As Variant 

Dim r2 As Variant 

Dim r As Variant 

Dim vw As Variant 

Dim vl As Variant 

Dim q As Variant 

Dim a As Variant 

Dim b As Variant 

Dim t As Variant 

Dim s_bar As Variant 

vw= Rangefvw"). Value 

vl = Range(’’vl’’).Value 

a = Range(’’alpha’’).Value 

b = RangeC’beta"). Value 

m = Range(’’m’’).Value 

q = RangeC’q"). Value 

SheetsC’Results"). Select 

Range(’’s_bar"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 
Rangeft"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 
Range(’’E_G2"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 
Fort = 0 To m 
Rangeft"). Select 
ActiveCell.Offset(t, 0). Value = t 

r1 = -b - 0.5 + 0.5 * (1 + ((4 * vw * (a + b + t + 1) • (a + b + t)) / (vw + vl * q))) 0.5 

r2 = -b - 0.5 - 0.5 * (1 + ((4 * vw * (a + b + t + 1) * (a + b + t)) / (vw + vl * q))) 0.5 

If ((r1 <= 0) And (r2 <= 0)) Then GoTo Reject_t 

If (((r2 > 0) And (r1 > 0)) And (r1 > r2)) Then r = r1 

If (((r2 > 0) And (r1 > 0)) And (r2 > r1)) Then r = r2 

If ((r2 > 0) And (r1 <= 0)) Then r = r2 

If ((r1 > 0) And (r2 <= 0)) Then r = r1 
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s_bar = Application. Max(0, (t - Application. RoundDown(r, 0))) 

EG2 = 0 

For j = s_bar To t 

bJ1 = Exp(Application.GammaLn(a + b)) * Exp(Application.GammaLn(a + j)) * 
Exp(Application.GammaLn(b + t - j)) 

bJ2 = Exp(Application.GammaLn(a)) * Exp(Application.GammaLn(b)) *_ 
Exp(Application.GammaLn(a + b + t)) 
bJ3 = Application. Combin(t, j) 
bj = bJ1 / bJ2 * bJ3 

EG2_1 = vw - ((vw + vl * q) * (b + t - j + 1) * (b + t - j) _ 

/ ((a + b + t + 1) * (a + b + t))) 

EG2 = EG2 + (EG2_1 * bJ) 

Next j 

EG2 = Application. Max(EG2 * (m - 1) / 2, 0) 

Range("E_G2").Select 
ActiveCell.Offset(t, 0).Value = EG2 
ActiveCell.Offset(t, 1). Value = s_bar 
GoTo next_t 
Reject_t: 

Range("E_G2"). Select 
ActiveCell. Offset (t, 0). Value = 0 
ActiveCell.Offset(t, 1). Value = 0 
next_t: 

Next t 

Dim check_range As Range 
Dim optimal_t As Integer 
Dim t_counter As Integer 
Dim optimal_gain As Variant 
SheetsC'Resu Its"). Select 
Range("B2") .Activate 

Range(ActiveCell, ActiveCell. End(xlDown)). Select 

Set check_range = Selection 

Range("max_gain").Select 

ActiveCell.Value = Application.Max(check_range) 

optimal_gain = Application. Max(check_range) 

Range("b2") .Activate 
Fort_counter = 1 To m + 1 

If ActiveCell.Value = optimal_gain Then 
optimal_t = ActiveCell. Offset (0, -1). Value 
GoTo loop_exit_4 
End If 

ActiveCell.Offset(1 , 0).Activate 
Next t_counter 
loop_exit_4: 

Range("t_star"). Value = optimal_t 

t 

This routine calculates the risks associated with accepting the 
'system after testing. 

I 

Dim t_star As Variant 
Dim s_bar_star As Variant 
Dim p_bar As Variant 
Dim p_bar1 As Variant 
Dim p_bar2 As Variant 



39 



t_star = Range("t_star")-Value 

r1_star = -b - 0.5 + 0.5 * (1 + ((4 * vw * (a + b + t_star + 1) * _ 

(a + b + t_star)) / (vw + vl * q))) '' 0.5 
r2_star = -b - 0.5 - 0.5 * (1 + ((4 * vw * (a + b + t + 1) * _ 

(a + b + t)) / (vw + vl * q))) 0.5 

If (((r2_star > 0) And (r1_star > 0)) And (r1_star > r2_star)) Then r_star = r1_star 

If (((r2_star > 0) And (r1_star > 0)) And (r2_star > r1_star)) Then r_star = r2_star 

If ((r2_star > 0) And (r1_star <= 0)) Then r_star = r2_star 

If ((r1_star > 0) And (r2_star <= 0)) Then r_star = r1_star 

s_bar_star = Application. Max(0, (t_star- Application. RoundDown(r_star, 0))) 

Range("s_bar_star").Value = s_bar_star 

m_t = m - t_star 

N_mt = Application. RoundDown(m_t / 2, 0) 
p_bar = 0 
p_bar1 = 0 
p_bar2 = 0 

Sheets("Results"). Select 
Range("PM").Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 

Selection.Clear 

Range("k"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 

Selection.Clear 

RangeC'PMD"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 

Selection.Clear 

SheetsfResults"). Select 

Range("p_bar"). Select 

Selection.Clear 

If ((t_star = 0) And Not (optimal_gain = 0)) Then 
GoTo NoTest 

Elself (optimal_gain = 0) Then 
GoTo Reject 
Else 

For k = 0 To N_mt 
PM1 = 0 
PM2 = 0 

For] = s_bar_starTo t_star 
PM1a = 0 
For n = 0 To k 

PM1a = PM1a + ((Application. Combin(t_star, j)) *_ 

(Application. Combin(N_mt, k)) * _ 

(Application. Combin(k, n)) * _ 

((-1)^n)*_ 

Exp(Application.GammaLn(a + ]))*_ 

Exp(Application.GammaLn(2 * N_mt - 2*k + 2*n + b + t_star - j)) / . 
Exp(Application.GammaLn(2 * N_mt - 2*k+2*n + a + b + t_star))) 

Next n 

PM1 = PM1 + PM1a 

PM2 = PM2 + ((Application. Combin(t_star, ])) *_ 

Exp(Application.GammaLn(b + t_star- ])) * _ 

Exp(Application.GammaLn(a + j)) /_ 

Exp(Application.GammaLn(a + b + t_star))) 

Next j 
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PM = PM1 / PM2 
SheetsC'Results"). Select 
Range("PM").Select 
ActiveCell.Offset(k, 0) .Activate 
ActiveCell.Value = PM 
Next k 

RangeC'PMD"). Select 
ActiveCell.FormulaR1C1 = "=RC[-1]" 

For i = 1 To N_mt 
ActiveCell.Offset(1, 0).Activate 
ActiveCell.FormulaR1C1 = "=RC[-1]+R(-1]C" 

Next i 

Range("k"). Select 
For i = 0 To N_mt 
ActiveCell.Value = i 
ActiveCell.Offset(1, 0).Activate 
Next i 

For j = s_bar_star To t_star 

p_bar1 = p_bar1 + ((1 - ((b + t_star - j + 1) * (b + t_star - J) / 
((a + b + t_star + 1) * (a + b + t_star)))) * _ 
Application.Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammaLn(b + t_star - j))) 
p_bar2 = p_bar2 + ((Application. Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammaLn(b + t_star - j)))) 

Next] 

p_bar = (p_bar1 / p_bar2) 

Range("p_bar"). Select 

ActiveCell.Value = p_bar 

Range(''p_bar_comment"). Select 

Selection. Clear 

End If 

GoTo Fini 

Reject; 

Ra ng e ("t") .Activate 
For i = 0 To m 

ActiveCell.Offset(i, 0).Value = i 
ActiveCell.Offset(i, 1). Value = 0 
Next i 

Range("max_gain").Value = 0 
Range("t_star"). Select 
Selection. Clear 
MsgBox "Reject System" 

Range("s_bar"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 

Range("s_bar_star").Value = "NA" 

SheetsC'Results"). Select 
RangeC'k"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 

Selection.Clear 

Range("p_bar"). Select 

Selection.Clear 
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Range("p_bar_comment"). Select 
Selection. Clear 
Range("PMD").Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 
RangefPM"). Select 

Range(ActiveCell, ActiveCell.End(xlDown)). Select 
Selection. Clear 
ActiveCell.Value = "REJECT" 

ActiveCell.Offset(0, 1).Activate 
ActiveCell.Value = "SYSTEM" 

GoTo Fini 
NoTest: 

MsgBox "Optimal Gain Achieved \with No Testing" 

Sheets("Results").Select 
Range("p_bar"). Select 
Selection.Clear 

ActiveCell.Offset(0, 1). Value = Prior probability used since 0 tested" 

N_m = Application. RoundDown(m / 2, 0) 

Rangefk"). Select 
For k = 0 To N_m 
PM1 =0 
PM2 = 0 
s_bar_star = 0 
t_star = 0 

For j = s_bar_star To t_star 
PM1a = 0 
For n = 0 To k 

PM1a = PM1a + ((Application.Combin(t_star, j)) *_ 

(Application. Combin(N_mt, k)) * _ 

(Application. Combin(k, n)) *_ 

((-1)^n)*_ 

Exp(Application.GammaLn(a + j)) * _ 

Exp(Application.GammaLn(2 * N_mt - 2*k+2*n + b + t_star - j)) / . 
Exp(Application.GammaLn(2 * N_mt - 2*k + 2*n + a + b + t_star))) 

Next n 

PM1 = PM1 + PM1a 

PM2 = PM2 + ((Application. Combin(t_star, j)) *_ 

Exp(Application.GammaLn(b + t_star - j)) * _ 

Exp(Application.GammaLn(a + j)) /_ 

Exp(Application.GammaLn(a + b + t_star))) 

Next] 

PM = PM1 /PM2 
SheetsfResults"). Select 
RangeC'PM"). Select 
ActiveCell.Offset(k, 0).Activate 
ActiveCell.Value = PM 
Next k 

RangefPMD"). Select 
ActiveCell.FormulaR1C1 = ”=RC[-1]" 

For i = 1 To N_mt 

ActiveCell.Offset(1 , 0).Activate 
ActiveCell.FormulaR1C1 = "=RC[-1]+R[-1]C" 

Next i 
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RangeC'k"). Select 
For i = 0 To N_m 
ActiveCell.Value = i 
ActiveCell.Offset(1 , 0).Activate 
Next i 

For j = s_bar_star To t_star 

p_bar1 = p_bar1 + ((1 - ((b + t_star - j + 1) * (b + t_star - j) / _ 

((a + b + t_star + 1) * (a + b + t_star)))) * _ 
Application. Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammaLn(b + t_star - j))) 
p_bar2 = p_bar2 + ((Application. Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammaLn(b + t_star - j)))) 

Next j 

p_bar = (p_bar1 / p_bar2) 

Range("p_bar").Select 
ActiveCell.Value = p_bar 
Fini; 

Range("G5") .Activate 
End Sub 

I 

This routine provides for the functions that occur on workbook opening. 

I 

Sub Auto_openO 
Call getjnput 

ActiveWorkbook.Protect contents = True 
End Sub 



43 




44 



APPENDIX E. INSTALLATION INSTRUCTIONS 



The installation diskette has the following three files; 

1 . Gl.xls - the Excel workbook which contains the algorithm for analysis of 
the single weapon case. 

2. G2.xls - the Excel workbook which contains the algorithm for analysis of 
the two- weapon salvo case. 

3. Readme.txt - an ASCII text file which contains the text found in this 
appendix. 

To use the analysis files, simply open them with Excel V7.0 or later and follow the 
dialog box prompts. Input parameters can be changed and the algorithm repeated by 
clicking the button on the worksheet marked “Press to Change Parameters.” The 
workbooks open recommending a “read-only” format. This is done to protect the 
structure of the “Results” worksheet. The worksheets which contain the input dialog box 
and the source code are “hidden.” The source code is also “protected” and cannot be 
altered. If one wishes to alter the code to develop the ideas of the software further, cut 
and paste the appropriate code into a different workbook and proceed. 
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